[core.logging] Deprecates legacy logging dest, json, verbosity and rotate configurations #94238

TinaHeiligers · 2021-03-09T23:35:00Z

Summary

Resolves #82431.
Closes #86125

This PR deprecates legacy logging configuration in favor of the new KP logging system.
These include:

logging.silent
logging.quiet
logging.verbose
logging.dest
logging.json
logging.rotate & sub-options: enabled, usePolling, pollingInterval, everyBytes, keepFiles
logging.events
logging.filter

The deprecation warnings shown when legacy logging is configured include a short description about KP logging config alternatives and link to the KP logging README. These links will be changed to reference the user-facing docs once #57502 is addressed.

The deprecations also apply to any configuration passed in as cli arguments except for the short-hand versions of --verbose and --silent logging levels.
--verbose is now handled slightly differently and will do the following:

When running in dev mode (env === 'development), deprecation warnings for using --json won't be shown (we explicitly set --logging.json in dev cli).
When KP logging is configured, sets KP root log level to all
When KP isn't configured, sets legacy log level to verbose (logging.verbose = true)

Release Notes

Deprecates legacy logging configuration in favor of the new Kibana Platform logging system. For example, deprecates logging.json and logging.rotate.*.

--verbose will replace legacy-format logs with Kibana platform log format if logging.root.appenders is configured and won't show a deprecation warning. If Kibana platform logging is not configured, --verbose sets logging.verbose: true and will warn of the deprecated configuration.

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials (these will be added in Add user-facing docs for new logging configuration #57502)
Unit or functional tests were updated or added to match the most common scenarios

For maintainers

This was checked for breaking API changes and was labeled appropriately

docs/migration/migrate_8_0.asciidoc

src/core/server/config/deprecation/core_deprecations.ts

src/core/server/config/integration_tests/config_deprecation.test.ts

src/core/server/config/deprecation/core_deprecations.ts

TinaHeiligers

Self review.

src/core/server/config/deprecation/core_deprecations.ts

src/core/server/logging/README.mdx

lukeelmers

@TinaHeiligers One thing that occurred to me when summarizing these changes for the beats team is that we might want to mention in the docs that switching to the new logging config is going to mean you start seeing duplicate log entries in both formats.

Most users will have not run into this yet, but if they are ingesting their Kibana logs somewhere, it would be good for them to know that using the new config will result in additional log entries which are in a different structure.

…ions

…et legacy logging level when not opted in to KP logging

elasticmachine · 2021-03-15T20:57:10Z

Pinging @elastic/kibana-core (Team:Core)

lukeelmers

One question I want to raise is whether we should consider adding deprecation notices for the undocumented logging.filter, logging.events.log, and logging.events.error... AFICT those are the only remaining legacy logging configs that would not be deprecated once this PR is merged.

On the one hand, as they were never officially documented, there should be no expectation of them not breaking. But on the other hand, if someone stumbled upon these settings, it would make their lives easier to get a deprecation notice warning them when they are in use. WDYT?

src/core/server/config/deprecation/core_deprecations.ts

lukeelmers · 2021-03-16T18:28:38Z

src/core/server/config/deprecation/core_deprecations.ts

+  ) {
+    log(
+      '"logging.rotate" and sub-options have been deprecated and will be removed in 8.0. ' +
+        'Moving forward, you can enabled log rotation using the "rolling-file" appender for a context ' +


enabled -> enable is a typo
context -> logger is a nit (In #90764 we switched from labeling loggers with context to just using name... but there are probably a few other places in the logging docs where we'd need to update this, so no strong feelings on it).

Suggested change

'Moving forward, you can enabled log rotation using the "rolling-file" appender for a context ' +

'Moving forward, you can enable log rotation using the "rolling-file" appender for a logger ' +

docs/migration/migrate_8_0.asciidoc

src/cli/serve/serve.js

src/core/test_helpers/kbn_server.ts

src/legacy/server/logging/index.js

src/legacy/server/kbn_server.js

src/cli/serve/serve.js

TinaHeiligers · 2021-03-16T23:04:30Z

One question I want to raise is whether we should consider adding deprecation notices for the undocumented logging.filter, logging.events.log, and logging.events.error... AFICT those are the only remaining legacy logging configs that would not be deprecated once this PR is merged.

On the one hand, as they were never officially documented, there should be no expectation of them not breaking. But on the other hand, if someone stumbled upon these settings, it would make their lives easier to get a deprecation notice warning them when they are in use. WDYT?

@lukeelmers This is a tough call. I'm not apposed to helping folks out if we have an alternative in the KP logging system. We don't yet support filtering in the KP so I'd err on the side of not adding a deprecation warning for that. We could add deprecation warnings for logging.events.log and logging.events.error (what's the expected value type for these, i.e. logging.event.error: ????) since we do offer alternatives for those (logging level and all errors are automatically logged or one can explicitly set the root logger level to error).

However, I'm not sure if it would be more of a surprise for folks to start seeing these warnings only to have the settings removed completely in a few minors.

I'll add them and we can see what other reviewers think.

…ent.error configuration

src/legacy/server/logging/index.js

docs/migration/migrate_8_0.asciidoc

packages/kbn-legacy-logging/src/schema.ts

src/cli/serve/serve.js

src/core/server/config/deprecation/core_deprecations.ts

joshdover · 2021-03-17T12:49:31Z

This is a tough call. I'm not apposed to helping folks out if we have an alternative in the KP logging system. We don't yet support filtering in the KP so I'd err on the side of not adding a deprecation warning for that. We could add deprecation warnings for logging.events.log and logging.events.error (what's the expected value type for these, i.e. logging.event.error: ????) since we do offer alternatives for those (logging level and all errors are automatically logged or one can explicitly set the root logger level to error).

However, I'm not sure if it would be more of a surprise for folks to start seeing these warnings only to have the settings removed completely in a few minors.

I'll add them and we can see what other reviewers think.

I prefer we show warnings for all settings that will no longer work, even if they were never documented and/or don't have a replacement in 8.0. +1 on the warnings for all 3 of those configs.

joshdover

Looks really close! The only scenario that doesn't seem to be working quite right for me is the one where I have the KP logger configured (logging.root.appenders is set) and I'm using the --verbose flag. It's still showing a warning log message about logging.verbose. I left a comment below about how to fix the issue.

docs/migration/migrate_8_0.asciidoc

src/cli/serve/serve.js

src/core/server/config/deprecation/core_deprecations.ts

lukeelmers

Updates here LGTM, thanks for adding those logging.events and logging.filter deprecations too!

lukeelmers · 2021-03-17T21:04:38Z

src/core/server/config/deprecation/core_deprecations.ts

+  if (has(settings, 'logging.events.error')) {
+    log(
+      '"logging.events.error" has been deprecated and will be removed ' +
+        'in 8.0. Moving forward, you can use "logging.root.level: error" in your logging configuration. '


super nit: Could still link to the docs for both of the logging.events deprecations

@lukeelmers is it ok if we do that once the docs for the KP logging system are added and I go back and change the other links?

works for me 🙂

TinaHeiligers · 2021-03-17T21:30:28Z

@elasticmachine merge upstream

kibanamachine · 2021-03-17T23:40:26Z

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

💔 Build #113796 failed 45971d9
💔 Build #113686 failed 7850877
💚 Build #113560 succeeded 58aeff3
💚 Build #113168 succeeded f87ae2e
💚 Build #113124 succeeded 1adb68f

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

kibanamachine · 2021-03-17T23:52:25Z

💔 Backport failed

❌ 7.x: Commit could not be cherrypicked due to conflicts

To backport manually, check out the target branch and run:
node scripts/backport --pr 94238

…tate configurations (elastic#94238) Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> # Conflicts: # docs/migration/migrate_8_0.asciidoc # src/core/server/config/deprecation/core_deprecations.ts

…tate configurations (#94238) (#94878) Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> # Conflicts: # docs/migration/migrate_8_0.asciidoc # src/core/server/config/deprecation/core_deprecations.ts

TinaHeiligers added release_note:deprecation Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc v8.0.0 Feature:Logging v7.13.0 labels Mar 9, 2021

TinaHeiligers commented Mar 10, 2021

View reviewed changes

docs/migration/migrate_8_0.asciidoc Show resolved Hide resolved

TinaHeiligers commented Mar 10, 2021

View reviewed changes

src/core/server/config/deprecation/core_deprecations.ts Outdated Show resolved Hide resolved

TinaHeiligers commented Mar 10, 2021

View reviewed changes

src/core/server/config/deprecation/core_deprecations.ts Show resolved Hide resolved

TinaHeiligers commented Mar 10, 2021

View reviewed changes

src/core/server/config/integration_tests/config_deprecation.test.ts Show resolved Hide resolved

TinaHeiligers commented Mar 10, 2021

View reviewed changes

src/core/server/config/deprecation/core_deprecations.ts Show resolved Hide resolved

TinaHeiligers commented Mar 10, 2021

View reviewed changes

mshustov mentioned this pull request Mar 10, 2021

Deprecate LP log rotation in favour of Log rotation appender #86125

Closed

lukeelmers reviewed Mar 10, 2021

View reviewed changes

TinaHeiligers mentioned this pull request Mar 11, 2021

Add user-facing docs for new logging configuration #57502

Closed

TinaHeiligers added 6 commits March 12, 2021 11:18

Deprecates legacy logging dest, json, verbosity and rotate configurat…

aac1dc3

…ions

Adds logging rotate options to README

58df992

Adds log rotation to migration guide docs

2fa7fe0

Update config_deprecations jest integration tests

376e047

Adds cliArgs back

08e9cba

Fixes typos

abae294

TinaHeiligers force-pushed the logging/deprecate-legacy-config branch from 9639539 to abae294 Compare March 12, 2021 18:51

TinaHeiligers added 3 commits March 12, 2021 16:45

Retains support for cliArgs --verbose and --silent

a389f4e

Only log logging.json deprecation warnings when not in dev and only s…

b9011ff

…et legacy logging level when not opted in to KP logging

Updates snapshot

1adb68f

elastic deleted a comment from kibanamachine Mar 15, 2021

TinaHeiligers added 2 commits March 15, 2021 10:40

Removes server logs from legacy

63e4fe6

snapshot update

f87ae2e

elastic deleted a comment from kibanamachine Mar 15, 2021

TinaHeiligers marked this pull request as ready for review March 15, 2021 20:57

TinaHeiligers requested a review from a team as a code owner March 15, 2021 20:57

TinaHeiligers requested review from spalger, lukeelmers and joshdover March 15, 2021 20:57

lukeelmers reviewed Mar 16, 2021

View reviewed changes

Merge branch 'master' into logging/deprecate-legacy-config

f8dd77f

TinaHeiligers added 3 commits March 16, 2021 16:44

Handles initial review comments

84cf8fe

Adds deprecation warnings for legacy logging.event.log and logging.ev…

f1e901d

…ent.error configuration

Removes duplicate comment

58aeff3

pgayvallet reviewed Mar 17, 2021

View reviewed changes

joshdover reviewed Mar 17, 2021

View reviewed changes

docs/migration/migrate_8_0.asciidoc Show resolved Hide resolved

src/cli/serve/serve.js Outdated Show resolved Hide resolved

src/core/server/config/deprecation/core_deprecations.ts Show resolved Hide resolved

Deprecates logging.filter, addresses review comments

7850877

spalger approved these changes Mar 17, 2021

View reviewed changes

Updates deprecation warnings and snapshots

45971d9

lukeelmers approved these changes Mar 17, 2021

View reviewed changes

Merge branch 'master' into logging/deprecate-legacy-config

c6af872

TinaHeiligers added the auto-backport Deprecated - use backport:version if exact versions are needed label Mar 17, 2021

TinaHeiligers merged commit 28942df into elastic:master Mar 17, 2021

TinaHeiligers deleted the logging/deprecate-legacy-config branch March 17, 2021 23:50

TinaHeiligers mentioned this pull request Mar 18, 2021

[7.x] [core.logging] Deprecates legacy logging dest, json, verbosity and rotate configurations (#94238) #94878

Merged

TinaHeiligers mentioned this pull request Mar 25, 2021

[DOCS] Adds user-facing docs for the new KP logging configuration #94993

Merged

2 tasks

jbudz mentioned this pull request Aug 18, 2021

[deb/rpm] Deprecation warnings caused by legacy logger #109159

Closed

akash1810 mentioned this pull request Jan 24, 2022

RPM package sets deprecated logging.dest cli flag #111263

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[core.logging] Deprecates legacy logging dest, json, verbosity and rotate configurations #94238

[core.logging] Deprecates legacy logging dest, json, verbosity and rotate configurations #94238

TinaHeiligers commented Mar 9, 2021 •

edited

Loading

TinaHeiligers left a comment

lukeelmers left a comment

elasticmachine commented Mar 15, 2021

lukeelmers left a comment

lukeelmers Mar 16, 2021

TinaHeiligers commented Mar 16, 2021 •

edited

Loading

joshdover commented Mar 17, 2021

joshdover left a comment

lukeelmers left a comment

lukeelmers Mar 17, 2021

TinaHeiligers Mar 17, 2021

lukeelmers Mar 18, 2021

TinaHeiligers commented Mar 17, 2021

kibanamachine commented Mar 17, 2021

kibanamachine commented Mar 17, 2021

	'Moving forward, you can enabled log rotation using the "rolling-file" appender for a context ' +
	'Moving forward, you can enable log rotation using the "rolling-file" appender for a logger ' +

[core.logging] Deprecates legacy logging dest, json, verbosity and rotate configurations #94238

[core.logging] Deprecates legacy logging dest, json, verbosity and rotate configurations #94238

Conversation

TinaHeiligers commented Mar 9, 2021 • edited Loading

Summary

Release Notes

Checklist

For maintainers

TinaHeiligers left a comment

Choose a reason for hiding this comment

lukeelmers left a comment

Choose a reason for hiding this comment

elasticmachine commented Mar 15, 2021

lukeelmers left a comment

Choose a reason for hiding this comment

lukeelmers Mar 16, 2021

Choose a reason for hiding this comment

TinaHeiligers commented Mar 16, 2021 • edited Loading

joshdover commented Mar 17, 2021

joshdover left a comment

Choose a reason for hiding this comment

lukeelmers left a comment

Choose a reason for hiding this comment

lukeelmers Mar 17, 2021

Choose a reason for hiding this comment

TinaHeiligers Mar 17, 2021

Choose a reason for hiding this comment

lukeelmers Mar 18, 2021

Choose a reason for hiding this comment

TinaHeiligers commented Mar 17, 2021

kibanamachine commented Mar 17, 2021

💚 Build Succeeded

Metrics [docs]

History

kibanamachine commented Mar 17, 2021

💔 Backport failed

TinaHeiligers commented Mar 9, 2021 •

edited

Loading

TinaHeiligers commented Mar 16, 2021 •

edited

Loading